<h2 align=center>A program to attempt recovery of an x-file's contents</h2>
<p><dfn>X-Files</dfn> is an image filing system written by
<a href="mailto:andy@wonderworks.co.uk" subject="X-Files">Andy Armstrong</a>. Normal (filecore based) Risc OS filing systems are limited to 10 character leafnames and 77 files per directory. X-Files supports long filenames (up to 256 characters internally) and an unlimited number of files per directory, storing files in an x-file (which in other respects behaves like a "normal" directory).</p>
<p>Unfortunately X-Files is not totally robust, and can occasionally corrupt an x-file, which it will then refuse to open. Sod's law ensures that the data in the x-file is very important and not backed up anywhere else. This is where <code>x-recover</code> comes in; <code>x-recover</code> will <strong>attempt</strong> to make sense of and extract data from corrupted x-files.</p>
<h2 align=center><a name="!warranty">Important: x-recover comes with absolutely <blink>NO WARRANTY</blink></a></h2>
<p>This program is distributed in the hope that it will be useful, but <strong>without any warranty</strong>; without even the implied warranty of <strong>merchantability</strong> or <strong>fitness for a particular purpose</strong>. As <code>x-recover</code> is written in C it almost certainly breaks the rules somewhere, and hence is exhibiting "<dfn>undefined behaviour</dfn>". What this means in English is that it could do anything at all, including, but not limited to:
<ol><li>Working exactly as intended</li>
<li>Trashing your hard disc</li>
<li>Turning Barbara Cartland into a <a href="#goth">goth<sup>*</sup></a></li>
<p><code>x-file</code> is the pathname of the x-file to attempt to recover. <code>x-recover</code> does not check the filetype (so if <code>x-file</code> is not actually an x-file <code>x-recover</code> will generate copious warnings as it discovers this).</p>
<p><code>destination</code> is the pathname a directory (or empty x-file) to write recovered contents into. <code>destination</code> should exist, or use the <a href="#-c"><code>-c</code></a> option to create a new x-file with this name. <code>destination</code> can be omitted if file output is suppressed (<a href="#-g"><code>-g</code></a> or <a href="#-n"><code>-n</code></a> options).</p>
<h3>Options</h3>
<table><tr><th valign=top><a name="-a"><code>-a</code></a></th><td>extract the full chunk <strong>a</strong>llocated to the file, rather than the <a href="#size">length used</a>. Using this option causes the chunktable to be written out.</td></tr>
<tr><th valign=top><a name="-c"><code>-c</code></a></th><td><strong>c</strong>reate <code>destination</code> as an x-file if no directory/x-file of this name exists.</td></tr>
<tr><th valign=top><a name="-d"><code>-d</code></a></th><td>Output probable <strong>d</strong>irectories in raw binary form. See <a href="#rawdirs">directories</a>.</td></tr>
<tr><th valign=top><a name="-f"><code>-f</code></a></th><td>write out any <strong>f</strong>ree space between chunks. <code>x-recover -a -d -f -1</code> will output the entire x-file split into chunks, which if concatenated in numerical order will give the original x-file.<a href="#size"><sup>[see size]</sup></a></td></tr>
<tr><th valign=top><a name="-g"><code>-g</code></a></th><td><strong>g</strong>uess the location of the chunktable and the root directory. <code>x-recover</code> simply runs through the file looking areas that resemble the chunktable and the root directory. When used with the <code>-r</code> and <code>-t</code> options this allows recovery even when the file header is corrupted. This option suppresses all disc output.</td></tr>
<tr><th valign=top><a name="-n"><code>-n</code></a></th><td><strong>n</strong>o disc output. Integrity checks and errors are still reported to the screen.</td></tr>
<tr><th valign=top><a name="-r"><code>-r <offset></code></a></th><td>specify the offset of the <strong>r</strong>oot directory in <code>x-file</code>. Useful if the header becomes corrupted - see <a href="#header">header</a>.</td></tr>
<tr><th valign=top><a name="-s"><code>-s</code></a></th><td><strong>s</strong>uppress file output, but still create the directories (and free space) in <code>destination</code>. Mostly used for development purposes to quickly check that things are working without having to wait while files are copied, but <code>x-recover -1 -f -s</code> will only extract directories and free space between directories.</td></tr>
<tr><th valign=top><a name="-t"><code>-t <offset></code></a></th><td>specify the offset of the chunk<strong>t</strong>able in <code>x-file</code>. Useful if the header becomes corrupted - see <a href="#header">header</a>.</td></tr>
<tr><th valign=top><a name="-v"><code>-v</code></a></th><td><strong>v</strong>erbose info. <code>x-recover</code> prints out more information about what it is doing. Use more <code>v</code>s for more verbosity.</td></tr>
<tr><th valign=top><code>-1</code></th><td>Use <a href="#method1"><strong>Method 1</strong></a> to attempt to recover <code>x-file</code>'s contents. The various methods are described below.</td></tr>
<tr><th valign=top><code>-2</code></th><td>Use <a href="#method2"><strong>Method 2</strong></a> to attempt to recover <code>x-file</code>'s contents. Method 2 is the current default.</td></tr>
</table>
<h3>The x-file structure, and how this affects the prognosis</h3>
<p>As the x-file file contains within itself data from other files, it must also store information about the contained files and their data. If an x-file becomes corrupted, it is likely that some of this housekeeping information is lost. The x-file structure is described here - which parts survive determines if recovery is possible, and if so the fidelity achievable.
<tr><td>An x-file starts with a header which contains a signature<!-- (<code>XFIL</code>) -->, version information and pointers to the chunktable and root directory. If the information stored in the header becomes corrupted it is possible to search for the chunktable using the <a href="#-g"><code>-g</code></a> option.</td></tr>
<tr><td>Except for the header all information in the x-file is stored in <dfn>chunks</dfn>. The index containing chunk sizes and positions is stored in the <dfn>chunktable</dfn> - clearly if the chunktable is missing then the x-file is simply an amorphous lump of data (like a corrupt hard disc). Currently <code>x-recover</code> doesn't have the knowledge to identify files from inspection of contents, so if the chunktable is missing automated recovery is not possible as <code>x-recover</code> cannot determine where one file stops and the next starts. If the chunktable is reasonably intact then <a href="#method1">Method 1</a> can be used to extract the contents of each chunk as separate files, but all name, filetype and datestamp information will be lost.</td></tr>
<tr><td>Information about names, filetypes, attributes and modification time is stored in the <dfn>root directory</dfn> and its subdirectories. If the root directory can be found (either from the header or using the <a href="#-g"><code>-g</code></a> option) then <a href="#method2">Method 2</a> can be used to attempt recovery of the x-file's contents and directory structure, including file type information.</td></tr></table>
</p>
<h3><a name="method1">Method 1</a></h3>
<p>Method 1 reads in the chunktable, and then systematically writes out the contents of each non-free chunk as a file the destination directory/x-file. If it suspects that the chunk represents a <a href="#rawdirs">directory</a> it writes a text file describing the directory's contents, else it copies out the raw contents with filetype Data (FFD). All chunks can be copied out as raw contents with the <a href="#-d"><code>-d</code></a> option. Files and directories are named as <code>file0000 file0001</code> <i>etc</i> in the order that they are found in the x-file body (probably not the same order as chunktable - use <a href="#-v"><code>-vv</code></a> (<strong>v</strong>ery <strong>v</strong>erbose) or greater to list the chunktable).</p>
<h3><a name="method2">Method 2</a></h3>
<p>Method 2 reads in the chunktable and the root directory. Starting in the root directory it attempts to create a list of all files, their filetype, attributes, and location within the x-file (obtained via the chunktable). It recreates this directory structure within the destination directory/x-file, and then copies out all the files it found with their correct filenames, restoring filetypes, access attributes and modification times. It then tries to tally files that it knows exist but has no location with un-recovered chunks from the chunktable, and writes out any successful matches to the destination. Finally it writes the contents of any remaining unrecovered chunks as files in the destination using Method 1. Well, that's the plan...</p>
<h4><a name="rawdirs">Directories</a></h4>
<p>Like the normal Risc OS filing system, X-Files writes a special signature at the start of all directories so that when it reads the chunk containing the directory information, it can check that the information is not corrupted. X-Files' signature is the string <code>Andy</code><!-- what's wrong with Nick ? Risc OS uses Nick, and to me it seems a very nice name --> at the start of the chunk. Hence, if <code>x-recover</code> comes across a chunk about which it has no information, it will have a look for this signature. If present, the chunk is assumed to be a directory, which means that there is a small chance that a file which starts with the text <em>Andy</em> will interpreted as a directory and consequently garbled. If this happens, use the <a href="#-d"><code>-d</code></a> option to disable directory identification, and the file contents will be recovered intact.</p>
<h4><a name="size">Size</a></h4>
<p>Size of a file is stored <strong>twice</strong> in an x-file:
<ol><li>The directory stores the size of a file</li>
<li>The chunktable stores the size of a chunk</li></ol>
As the directory also stores an index into the chunktable, it is possible to have conflicting sizes reported for a given file. In this case <code>x-recover</code> will use the larger of the size in the directory and the size in chunktable. This could cause the recovered file to acquire a copy of the start of the next chunk, which can mean that <em>the sum of the sizes of the recovered parts is greater than the size of the x-file</em>. You don't get any extra information - you just get some of it twice!</p>
<h3>Bugs</h3>
<p>We don't do bugs.</p>
<p>None <strong>known</strong> - everything works to the design. However, there are known design deficiencies (and planned improvements). Please report bugs (preferably with fixes) to <a href="mailto:bagpuss@done.net" subject="x-recover"><i><bagpuss@done.net></i></a>.
If you can supply an x-file to demonstrate then this would be useful. Currently I'm quite happy for relevant e-mail up to 1Mb, but if <a href="http://bagpuss.done.net/">bagpuss.done.net</a> is up then use anonymous ftp to upload problem files to <a href="ftp://bagpuss.done.net/">ftp://bagpuss.done.net/</a></p>
<p>Files up to 100Mb are acceptable by this method. No, I'm not confusing Kb with Mb. If you're on Janet then you should be able to shift 100Mb to me in 7 minutes.</p>
<h4>Design deficiencies</h4>
<ol><!-- Fixed it <li>Options need to be separated by a space:<br>
Fixed this - fix your web browser so that it understands the comment sequence 8-) -->
<li>Unrecovered chunks are written out as <code>file####</code> in the destination directory/x-file <em>after</em> files are recovered, overwriting any genuine files with the same name(s). Of course, no-one names files like this...
<br>(So if you recover one x-file into another <em>rename these files rapidly</em>.)<!-- (or explore the <code>-m$</code> option) --></li>
<li>The <code>file####</code> naming system assumes that you have less than 10000 chunks. If this is violated <code>x-recover</code> will probably crash from <a href="#!warranty">undefined behaviour</a> as several internal buffers overflow. Don't say that I didn't warn you - this software comes with <a href="#!warranty"><strong>absolutely no warranty</strong></a>.</li>
<li>Method 2 doesn't re-attach subdirectories (and hence cannot recover name/date/attribute information for files contained in these directories). The chunks are recovered by Method 1. I can't just stick them in the destination directory/x-file in case two names clash, as the second will overwrite the first. Method 3 (when written (<a href="mailto:bagpuss@done.net" subject="x-recover Method 3">when someone asks me to</a>)) will hook unclaimed directories into the correct parent (with names like <code>dir_0000</code> ).</li>
<li>Likewise, Method 2 (and 3) ignore entries in the dirhash that don't correlate with full filenames. Consider what would happen with two files <code><strong>aard</strong>vark</code> and <code><strong>aard</strong>wolf</code>...</li>
</ol>
<h4>Don't be caught out by</h4>
<p>Method 2 restores file permissions (where known). If it restores a file as <code>LR/</code> (no write access) and an attempt is made to recover the x-file to the same directory, <code>x-recover</code> will not be able to open the file again, so will recover the contents as a Method 1 unrecovered chunk.</p>
<h2>Current version</h2>
<p>If there is newer version of <code>x-recover</code> it is probably <a href="http://www.liv.ac.uk/~nickc/">roughly here</a>. If it's missing (or deceased <!-- No no! 'E's pining! -->) <a href="mailto:bagpuss@done.net" subject="Your web pages are pining for the fjords">e-mail me!</a></p>
<hr>
<p><a name="goth"><strong>*</strong> Black is a much nicer colour than pink</a></p>
<!--footerrule-->
<hr>
<p><!--/footerrule--><!--footer0-->This page was last reviewed on <!--revdate--><a href="/~nickc/footers.html">Wednesday November 6<sup>th</sup> 1996</a><!--/revdate--><br>
